Papers with vocabulary pruning
TextPruner: A Model Pruning Toolkit for Pre-Trained Language Models (2022.acl-demo)
Copied to clipboard
| Challenge: | Large pre-trained language models have been used for many NLP tasks but computational resources are limited. |
| Approach: | They propose an open-source model pruning toolkit for pre-trained language models . they propose a self-supervised pruning method that can be applied without labeled data. |
| Outcome: | The proposed pruning method reduces model size without retraining the model and speeds up inference speed on the common CPU and GPU devices. |
Transferable Neural Projection Representations (N19-1)
Copied to clipboard
| Challenge: | Neural word embeddings require lookup and a large memory footprint making it hard to deploy on-device. |
| Approach: | They propose a skip-gram based architecture coupled with Locality-Sensitive Hashing projections to learn efficient dynamically computable representations. |
| Outcome: | The proposed model performs better than previous models on multiple NLP tasks. |